The parXXL Environment: Scalable Fine Grained Development for Large Coarse Grained Platforms

We present a new integrated environment for cellular computing and other fine grained applications. It is based upon previous developments concerning cellular computing environments (the ParCeL family) and coarse grained algorithms (the SSCRAP toolbox). It is aimed to be portable and efficient, and at the same time to offer a comfortable abstraction for the developer of fine grained programs. A first campaign of benchmarks shows promising results on clusters. 1 Motivations and Objectives Nowadays, many research areas consider multi-scale simulations based on ab initio computations: they aim to simulate complex macroscopic systems at microscopic level, using fundamental physical laws. Obviously, huge amounts of CPU are mandatory to run these simulations. Modern supercomputers and Grids, with large and scalable number of powerful processors, are interesting architectures to support these simulations. However designers and developers of algorithms and code for large scale applications are often confronted with a paradoxical situation: their modeling and thinking is fine-grained, speaking e.g. of atoms, cells, items, protein bases and alike, whereas modern computing architectures are coarse-grained providing few processors (up to several thousands ≈ 103) to potentially huge amount of data (thousands of billions of bytes ≈ 1012) and linking a substantial amount of resources (memory in particular) to each processor. Only few tools (for both, modeling and implementation) are provided to close this gap in expectation, competence and education. This article introduces the parXXL development environment, specially designed to close this gap between fine-grained modeling and coarse-grained computing architectures. It stems from two previous research projects that have In memoriam to our colleague Amelia De Vivo who passed away during the PARA2006 conference on June 21, 2006, in Ume̊a, Sweden. B. K̊agström et al. (Eds.): PARA 2006, LNCS 4699, pp. 1094–1104, 2007. c © Springer-Verlag Berlin Heidelberg 2007 The parXXL Environment: Scalable Fine Grained Development 1095 investigated optimization of computing resources (CPU, memory, communications, synchronization . . . ) and cellular oriented programming (to implement fine-grained models). Some collaborations with researchers in optic components and hot plasma (from LMOPS and LPMIA laboratories) guide parXXL design to an ease-to-use tool, and allow to identify collateral challenges. For example, some hot plasma simulation codes of our partners have been specially designed for global shared memory parallel computers. Intensive computation steps are split by optimized data rearrangement operations inside the global shared memory. But on large distributed architectures this kind of operations would be prohibitive. Some codes need to be re-designed and based on local computations, in order to support efficient runs on large distributed memory architectures and straightforward fine-grained implementations with parXXL. We are convinced that this algorithmic and programming methodology is required to achieve large multi-scale simulations.